Deformed grid based intra-operative system and method of use

ABSTRACT

The subject of this invention is a system and method for distortion adaptation for use with an imaging grid alignment apparatus (analog or digital) and method of intra-operative use for joint replacements, spine, trauma fracture reductions and deformity correction and implant placement/alignment. The system provides for real time dynamic position tracked distortion-adaption grid.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. provisional patentapplications 62/269,698 filed Dec. 18, 2015 and U.S. provisional patentapplication 62/357,293 filed Jun. 30, 2016, under 35 U.S.C. Sec. 119(e)(hereby incorporated by reference in its entirety)

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

None.

The names of the parties to a joint research agreement. OrthogridSystems, S.A.R.L., Orthogrid Systems, Inc., Richard Boddington andMediLux Capital Holdings, S.A.R.L.

Reference to a “Sequence Listing,” a table, or a computer programlisting appendix submitted on a compact disc and anincorporation-by-reference of the material on the compact disc: None.

FIELD OF THE INVENTION

The subject of this invention is a system for distortion adaptation foruse with radiographic grid alignment apparatus (analogue or digital) andmethod of intra-operative use for joint replacements, spine, traumafracture reductions and deformity correction and implantplacement/alignment.

BACKGROUND OF THE INVENTION

Many of the radiographic parameters essential to total hip arthroplasty(THA) component performance, such as wear, and stability, can beassessed intraoperatively with fluoroscopy. However even withintraoperative fluoroscopic guidance, the placement of an implant or thereduction of a bone fragment may still not be as close as desired by thesurgeon. For example, mal-positioning of the acetabular component duringhip arthroplasty can lead to problems. For the acetabular implant to beinserted in the proper position relative to the pelvis during hiparthroplasty requires that the surgeon know the position of thepatient's pelvis during surgery. Unfortunately, the position of thepatient's pelvis varies widely during surgery and from patient topatient. During trauma surgery, proper fracture management, especiallyin the case of an intra articular fracture, requires a surgeon to reducethe bone fragment optimally with respect to the original anatomy inorder to: provide the anatomical with joint the best chance torehabilitate properly; minimize further long term damage and, ifpossible, to regain its normal function. Unfortunately, in a fracturescenario, the original anatomical position of these bone fragments hasbeen compromised and their natural relationship with the correct anatomyis uncertain and requires the surgeon to use his/her best judgment inorder to promote a successful repair and, subsequent positive outcome.

Various devices are known to reduce mal-positioning of these surgicalcomponents. For example, a transverse acetabular ligament has beensuggested as a qualitative marker of the orientation of the acetabulum,(Archbold H A, et al. The Transverse Acetabular Ligament; an Aid toOrientation of the Acetabuiar Component During Primary Total HipReplacement: A Preliminary Study of 200 Cases InvestigatingPostoperative Stability, J Bone Joint Surg B R. 2006 July; 88(7):883-7).However, it has been suggested that the acetabulum may be deteriorateddue to arthritis. Others have proposed using a tripod device that usesthe anatomy of the ipsilateral hemi pelvis as the guide to position theprosthetic acetabular component. U.S. Patent Publication Number20090306679. This instrument has three points. The first leg ispositioned in the area of the posterior inferior acetabulum, a secondleg is positioned in the area of the anterior superior iliac spine and athird leg is positioned on the ileum of the subject. U.S. PatentPublication Number 20090306679. Regarding fracture fixation, or acorrection of a deformity or malunion, various devices have also beensuggested to support proper reconstruction and/or reduction of bonefragments. For example, a distal radius volar fixation plate has beensuggested to act as an invasive, intraoperative quantitative supportingbuttress to fix and provide a reference to the surgeon in order to helprealign the broken bony anatomy.

Distortion in a radiographic image, particularly fluoroscopic images, isa well-known phenomenon (Jareš V. The effect of electron optics on theproperties of the x-ray image intensifier. Adv Electron Elect Phys.10-9-85; 64(8): 549-59). Several distortion correction techniques havebeen published to date, for example by Kedgely A E, et al, in J ApplClin Med Phys. 2012 Jan. 5; 13(0:3441. doi: 10.1120/Jacmp.v13i1.3441.Image intensifier distortion correction for fluoroscopic RSA: the needfor independent accuracy assessment. These methods attempt to correctthe distortion using a series of beads to calibrate the amount ofdistortion in an image and then attempt to correct that distortion to anon-distorted view of the image. A need exists in the industry to allowfor intraoperative live adaptation of a grid (analogue, virtual,augmented, hologram, or 3D shape model) for use in measurements,positioning of implants and alignment of anatomy and to providedistortion adaptation with a radiographic grid alignment device.

SUMMARY OF THE INVENTION

The present subject matter includes a distorted dimensioned gridalignment device made from a distortion calibration array, wherein thevisualization of the dimensioned grid is deformed to accurately matchthe quantitative warp amount within an anatomical medical image. Thepresent subject matter includes a method to correct for distortion of ananatomical image captured from an imaging system by registering ananatomical image to a dimensioned grid by selecting at least oneanatomical landmark to provide a dimensioned grid with at least one gridindicator; calibrating the dimensioned grid to the image of a patient'sanatomy to provide a calibrated dimensioned grid; and deforming thecalibrated dimensioned grid to correct for the distortion of theanatomical image to generate a deformed calibrated dimensioned gridimage. The method further includes the step of: intra-operativelycomputing a point of interest in the image of a patient's anatomy withreference to the deformed calibrated dimensioned grid.

Another aspect of the present subject matter involves: a grid basedintraoperative system made of a non-transitory computer-readable storagemedium encoded with computer-readable instructions which form theapplication software and a processor to process the instructions,wherein the application software includes: a data capture moduleconfigured to capture and store a digital image acquired by an imagingdevice; a grid registration module configured to register an anatomicalimage to the grid; a distortion adaptation module configured to adaptthe grid to correct for the distortion in the anatomical image; and anoutcome module configured to provide at least one visual and/or auditoryinstruction intraoperatively to a user. The data capture module can beconfigured to obtain data from a sensor. The sensor can be in electroniccommunication with an augmented or mixed reality grid or trackable. Theoutcome module in one embodiment of the invention can be an augmentedreality display and the outcome module can be a 3D shape model orholographic display.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The drawings show schematically a fluoroscopic alignment plate apparatusand method of use according to an example form of the present invention.The invention description refers to the accompanying drawings:

FIG. 1 shows a schematic over-view of an intraoperative deformed gridsystem.

FIG. 2 shows a schematic overview of the software flow diagram of thesystem.

FIG. 3 shows an exemplary embodiment of a grid.

FIG. 4 shows an over-view schematic flow chart of the distortioncorrection method according to the present invention.

FIG. 5A is a screen shot image of a calibration grid and FIG. 5B is ascreen shot image of a calibration grid.

FIG. 6 shows a schematic overview of the distortion correction process.

FIGS. 7A and 78 are screen shot images of the user view of thedistortion correction step.

FIG. 8 shows an overview of the software flow diagram of theauto-segmentation process.

FIG. 9 shows exemplary view of a screen shot of the auto-segmentationprocess.

FIG. 10 shows an exemplary view of a head-up display view of the system.

DETAILED DESCRIPTION OF THE INVENTION

The present invention may be understood more readily by reference to thefollowing detailed description of the invention. It is to be understoodthat this invention is not limited to the specific devices, methodsconditions or parameters described herein and that the terminology usedherein is for the purpose of describing particular embodiments by way ofexample only and is not intended to be limiting of the claimedinvention. Also, as used in the specification including the appendedclaims, the singular forms “a,” “an,” and “the” include the plural, andreference to a particular numerical value includes at least thatparticular value, unless the context dearly dictates otherwise. Rangesmay be expressed herein as from “about” or “approximately” oneparticular value and/or to “about” or “approximately” another particularvalue. When such a range is expressed, another embodiment includes fromthe one particular value and/or to the other particular value.Similarly, when values are expressed as approximations, by use of theantecedent “about,” it will be understood that the particular valueforms another embodiment. These and other aspects, features andadvantages, of the invention will be understood with reference to thedetailed description herein, and will be realized by means of thevarious elements and combinations particularly pointed out in theappended claims. It is to be understood, that both the foregoing generaldescription and the following detailed description of the invention areexemplary and explanatory of preferred embodiments of the inventions,and are not restrictive of the invention as claimed. Unless definedotherwise, all technical and scientific terms used herein have the samemeaning as commonly understood by one of ordinary skill in the art towhich this invention belongs.

The present disclosure begins with acquiring radiographic images, suchas an X-ray images, of a patient during a surgical procedure. DigitalX-ray images are acquired intra-operatively, during surgical procedures,such as joint replacements, and trauma fracture reductions and deformitycorrection and implant placement/alignment. The acquired image can beviewed or further processed on a display device. A grid alignment device(analogue or digital) can be used with digital radiographic, ultrasound,CT, or other imaging system images acquired intra-operatively tofacilitate alignment or placement of an implant during a surgicalprocedure.

One of the objectives of the current technology is to provide distortionadaptation process to allow the inter-operative use of digital images.When a surgeon is performing an operation, a grid based intra-operativesystem can operate in an intra-operative mode, whereby the surgeon canmake use of the system, as a guide for adjusting the placement of animplant in the patient or the correction of a deformity. The dimensionedgrid is used to define alignment parameters of anatomy and implantplacement in a patient. In its simplest form, the grid can be digitallydisplayed over the anatomy and the surgeon can visually determinealignment and positioning measurements. However, to correct fordistortion in an anatomical image, an alignment grid of geometricalshapes or lines similar to or matching certain anatomical features inspecific regions of interest in an anatomical image is used to establisha distortion adaptation of the grid alignment apparatus. By this method,the anatomy is aligned intra-operatively, with the distorted anatomicalgeometrical grid, in any musculoskeletal application, which allows forreal time intra-operative placement of implants relative to anatomy asguided by the distorted grid alignment device.

In an exemplary embodiment, a dimensioned grid is secured within thepathway of the radiographic beam (such as attached to the imageintensifier). The anatomical image may or may not have distortion, butregardless of the amount of distortion present in the image, thedimensioned grid will be distorted relative to the amount of distortionpresent in the image, thereby providing visual quantitative analysispossible on an distorted anatomical image with the distortion-adapteddimensioned grid properly overlaid upon diverse point of interests ofthe anatomical image.

The process allows for the acquisition of a pre-operative orintra-operative initial image and that are subjected to a transformationprocess, such as for example Affine Transformation, thereby adjustingthe anatomical points to rotation, translation, and scaling. Then usinganother processic transformation, such as B-spline metric of fit, thegrid lines of the dimensioned grid are adapted real-timeintra-operatively to fit the anatomy, thus producing distorted linescurving to match or fit the musculoskeletal anatomy or shape of theimplant or trauma.

Turning to FIG. 1, a grid or a distortion calibration array basedintraoperative system 100 for providing a distorted dimensioned grid,such as lines curving to match or fit the musculoskeletal anatomy or theshape of the implant, as shown in distorted intra-operative images, isprovided. The grid or distortion calibration array based intraoperativesystem 100 includes a computer workstation 104. The computer workstation104 is made of: an image data capture unit 106; an image data generatingunit 108; a non-transitory computer-readable image data storage medium110; a processor 112; and computer-readable instructions 120. Thenon-transitory computer-readable storage medium 110 is encoded withcomputer-readable instructions, which form the application software 120.The computer workstation 104 also includes an input device 118 such as,a key board and a display 114, such as a monitor. The grid distortioncalibration array based intraoperative system 100 can optionally includea motion-tracking device 116. In an exemplary embodiment, an imagecapture unit 106; an image data generating unit 108 include measurementsand data can be acquired and displayed on AR/Holo lenses, glasses, orheads-up display.

The computer workstation 104 is electronically connected to an imagingsystem 220, such as for example C-arm and flat plate. An object, such asa patient 204 is positioned on a table 206 within the imaging system220. A distortion calibration array 200, such an analogue embodiment, isattached to an image intensifier 208 of the imaging system 220. Inanother embodiment, the dimensioned grid 200 allows the applicationsoftware 120 to generate a digital representation of the dimensionedgrid.

Turning to FIGS. 1 and 2, the application software 120 includes an imagecapturing module 300 that captures and stores the digital image acquiredby a imaging device 220. In one embodiment, the image is in the MOMimage format. The acquired image is sent to the distortion calibrationarray registration module 400 for registration. The registereddistortion calibration array is sent to the distortion adaptation module500 for adaption of the dimensioned grid. At least one visual and/orauditory instruction is then sent to an outcome module 600. This modulepresents at least one instruction for the surgeon to adjust the implantplacement and/or alignment until the surgeon is satisfied. Thefunctionalities of these software modules will be discussed in detailsdetail below.

In one embodiment, the application software 120 is organized intomodules and each module has at least one function black as shown in thisfigure. A hip alignment embodiment is used as an exemplary surgicalprocedure to illustrate the detailed operations of each block. It willbe appreciated that the distortion calibration array basedintraoperative system 100 disclosed here is not limited to perform onlythis surgical operation. For example, the present invention may beapplicable to other musculoskeletal applications such as arthroplastysurgery for hip, knee, spine, ankle and shoulder as well as traumasurgery for musculoskeletal repair. It will be clear to one skilled inthe art, that the present invention may be practiced with variations ofthe specific details described, hereinafter.

The application software 120 adapts the digital dimensioned gridpatterns in real-time intra-operatively to fit the subject's anatomy,thus producing distorted patterns to match or fit the musculoskeletalanatomy or shape of the implant or trauma and producing at least onevisual and/or auditory instruction or feedback intraoperatively to auser, such as a surgeon 202. The user 202 reviews the registeredanatomical image with the distortion adjusted dimensioned grid. Ifadjustment of the patient or the diagnostic equipment is required, thenthe surgeon selects points on the patient anatomy, inputs the selectedanatomical points into the work station, such as through the use of aninput device 112, or in an alternate auto-segmentation can be selectedby the surgeon. Once the points are inputted they become patternindicators for placement of the dimensioned grid.

Now referring to FIG. 3, in an exemplary embodiment, a dimensioned grid200 is shown. The dimensioned grid 200 can be either analogue orproduced digitally. With respect to an embodiment of an analoguedimensioned grid 200, it has a plurality of dimensioned radio-opaquelines, e.g. 230 relating to surgical variables. The portion of thedimensioned grid 200 that is not opaque is radiolucent. The dimensionedgrid 200 can include any shape or pattern of geometric nature or text toreference angles, length positioning or targeting. The dimensioned grid200 can be a single line, a geometrical pattern, number, letter or acomplex pattern of multiple lines and geometries that correspond tosurgical variables. The grid patterns can be predesigned or constructedintra-operatively in real-time based upon the surgeon's knowledge ofanatomy and clinical experience including interpretation of morphometricliterature and studies identifying key relationships and dimensionsbetween anatomical landmarks and its application in supporting goodsurgical technique as it relates to specific procedures. The analoguedimensioned grid 200 is used with an object being imagedintraoperatively with an imaging system 220, such as for example C-armand flat plate. With respect to a digital dimensioned grid, this form ofthe dimensioned grid 200 is generated by the application software 120.

FIGS. 1 and 4 show a detailed flow diagram of the method of thisinvention. In the data capture module 300, a digital image istransmitted to a computer workstation 104 with a display to project thedigital image. The digital image can be radiographic, MRI or anultrasound image. The digital image is taken at the command of thesurgeon or in one embodiment the digital image is continuously taken ata predefined interval during the entire operation.

The grid based intraoperative system 100 includes a display 114, such asa monitor screen, that is configured to display the digital image, otherrelevant text and graphic information to the surgeon. It can also becouple to an input device 112, such as keyboard, mouse, trackball orsimilar pointing devices, such as an infrared wand, as well as networkcommunication peripherals (not shown) so that it can communicate withother external servers and computer. In an exemplary embodiment, grid ordistortion calibration array based intraoperative system 100 receivesDICOM images from an imaging system 220, such as a C-Arm, from theoperative, side and DICOM images of the contra lateral side of thepatient. However, in some applications, such as surgery on a pelvis,only an x-ray image of one side is required. In this step of theprocess, an image of an anatomical portion of a patient is captured fromRadiographic/CT/MRI/Ultrasound (Step 410) in real time intraoperativelyor pre-operatively by a system that processes the image to provide ananatomical image model generation, saves the image and displays theimage on a visualization screen (Step 420).

In the grid registration module 400, a user 202, such as a surgeon,selects at least one anatomical landmark 310 in the anatomical image ona visualization screen. The display is visually available for thesurgeon by viewing on a display medium such as a computer/tabletmonitor, imaging system monitor, TV monitor, HUD, mixed reality glasses,augmented reality or holographic device such as glasses or contactlenses. The digital image can be 2D or a 3D generated shape model.Anatomical landmark 310 selection can be accomplished by a variousmethods including but not limited to: auto segmentation where thesoftware uses feature/pattern recognition process to auto-detect knownand targeted anatomical landmarks; use of remote infrared device, suchas a gyro mouse; voice command; air gestures; gaze (surgeon uses gazeand direction of eye or head motion) to control targeting or touchingthe visualization screen at the selected anatomical landmarks 310.

In one illustrative embodiment, the surgeon inputs the selection of theanatomical landmarks 310 to the workstation 104 manually or through theuse of a variety of input devices 112 such as, an infrared wand or anaugmented reality device. The application software registers adimensioned grid 200 with the selected anatomical landmarks 310. Themethod includes the step of registering an anatomical image to adimensioned grid by selecting at least one anatomical landmark 310 toprovide a dimensioned grid with at least one grid indicator (Step 430).

In the distortion correction module 500, a registration procedure isused to compute the deformation of the digital grid indicators. Theinitial step in the registration process is calibration (Step 440). Thesoftware identifies and recognizes calibration points 520 that areradiopaque in the image. These are of known dimensioned geometries. Agrouping of these points is a distortion calibration array 522. Thedistortion calibration array 522 is placed on the image intensifier 208or in the field of view of any imaging system 220 so that the knowndistortion calibration array lines/points are identified when an imageis taken and captured. These known patterns are saved for use in thedistortion adaptation/correction process (Step 450). The distortioncalibration array 522 is removed from visualization on the displaymedium in order to not obscure and clutter the image with unnecessarylines/points (Step 450).

Now referring to FIGS. 5A and 5B, distortion calibration array 522 isshown. A distortion calibration array 522 in this illustrativeembodiment is made a series of lines or points that are placed in orderto support the distortion adaptation of the dimensioned grid 200. Thedistortion calibration array points or lines 520 are radiopaque so thatthe distortion process can calculate the location of these points/lines522 relative to the anatomy and quantify the amount of distortion duringeach image taken. Once these points/lines are identified and used in thedistortion process, there is another process that removes thevisualization of these points/lines from the anatomical image so thatthey are not obstructing the surgeons view when the sees the dimensionedgrid 200 and the anatomical image.

In one embodiment, the registration process involves manually orautomatically detecting grid landmarks (such as grid line intersections,points, and line segments) on the dimensioned grid 200 superimposed onthe anatomical image and then aligning those landmarks via an affineregistration and a deformation field with corresponding landmarks on adistortion calibration array 522 of known geometry, which is representeddigitally. The method includes the step of deforming the calibrateddimensioned grid to correct for the distortion of the anatomical imageto generate a deformed calibrated dimensioned grid image. Knownradiopaque lines/points (from distortion calibration array 520) are usedto provide a measure of EM distortion in each anatomical image. Thedistortion is quantified and then the software generated virtual grid isadapted to match the distorted anatomy in each anatomical image 240(Step 460).

Now referring to FIGS. 6, 7A & 7B, the distortion calibration array 200is of non-uniform design, such that the selected anatomical landmarks310 are clustered more densely in regions of interest to the surgeon, inorder that the deformation correction may be estimated with greaterprecision in those regions. The deformation estimation proceeds asfollows: once selected anatomical landmarks 310 have been identified(either manually or automatically) on the array image, an AffineTransformation that produces the best mapping between correspondingselected anatomical landmarks 310 from the dimensioned grid 200 to thearray image is computed. Following transformation of the grid points bythe Affine Transformation, which adjusts the landmarks for translation,rotation, and scaling with respect to the array image landmarks in theDeformation Field (which is the residual difference between transformedgrid points and the array image points) is modeled using Thin-PlateSplines or any other suitable radial basis functions. Parameters of theThin-Plate Splines or radial basis functions are estimated by solving alinear system of equations.

Once the deformation field has been computed, the dimensioned grid isadapted in real-time intra-operatively to fit the patient anatomy, thusproducing a distorted grid indicator, such as lines curving that can beused to match or fit the musculoskeletal anatomy or the shape of theimplant. The deformation of the grid indicators is then applied inreal-time by first applying the Affine Transformation and then warpingthe grid indicators along the Deformation Field. A grid pattern basedupon the anatomical points that was defined and targeted in landmarkidentification is generated. The software program is configured tocompute the amount of distortion in each image and it quantifies thisamount relative to the anatomy/anatomy image and then displays thecalculated grid/Image relationship. displaying an image of the patient'sanatomy with the quantitatively distorted dimensioned grid image. Thesedeformed grids are tracked in real time with each new image taken. Thedeformed grid can be positioned relative to anatomy, implant, andfractures auto or manually by the user such as a surgeon (Step 470).

Numerous equations and formulas are used within the algorithms tocalculate: measurements, differences, angles, grid and implantpositions, fracture deviations to determine at least one measurement ofsurgical variables involving the implant or trauma (Step 480). In theoutput module 500, at least one measurement of surgical variablesinvolving the implant or trauma is presented to the user. Thisfacilitates the placement of the implant or treatment of the trauma inthe patient (Step 490). Once the preoperative plan is prepared andreviewed the surgeon positions the subject, such as a patient withrespect to the radiographic system, such as an x-ray device. The surgeonobtains a radiographic image, such as an x-ray of an area of interest.In the first step of the method, the radiographic images are generatedand transferred to the workstation with a display device. The anatomicalimage is displayed and surgeon reviews the radiographic image.Anatomical points are selected by the surgeon and then the grid isregistered to the image using image registration based on the selectedanatomical landmark points. Image registration is defined as an AffineTransformation of the grid to produce a best fit between predefinedlandmark points on the grid and their corresponding anatomical landmarkpoints. Affine Transform parameters that produce a best fit may bedefined as those parameters that produce the minimal least squaresdifferences between predefined landmark points and anatomical landmarkpoints, but may also include other optimization criteria. Followingimage registration, an estimation of the 3D pitch, and yaw of thepatient anatomy may be made using inter-anatomical landmark differencesand an assumption of the symmetry of the anatomy of the patient.

Measurements and data can also be sent to or communicated with a roboticsystem, haptic controlled device or touch/force sensor to amixed/augmented/holographic reality display showing visualization,alignment, and placement of instruments, bones or implants in 2D/3Dmixed/augmented/holographic reality image/shape model with, thedimensioned grid in a surgical environment in real-time intraoperativelyby projecting mixed/augmented reality grid data and image measurementsfor live mixed/augmented reality tracking of the dimensioned grid,surgical instruments and implant (step 495).

Now referring to FIGS. 8 and 9, the auto-segmentation module is shown.In auto-segmentation, the at least one anatomical landmark 310 selectedby the surgeon is automatically selected for each successive anatomicalimage. Auto-segmentation allows a surgeon to work more rapidly.Auto-segmentation is accomplished through a combination of one or moreof the following techniques: intensity thresholding; feature detectionon the intensity edges in the image, including shape detection via theHough Transform or other methods; feature detection followed byregistration of a 2D or 3D anatomical atlas with predefined landmarkpositions. With respect to the system, in one embodiment, theapplication software 120 is organized into modules and each module hasat least one function block as shown in this figure. Function 810includes automatic segmentation, function 820 includes automatic gridpoint selection, function 830 involves creating a grid from selectedpoints and function 840 involves an overlay of the grid on an anatomicalimage.

Now referring to FIGS. 1 and 10, an alternative embodiment of the system100 is shown. The system components include an input of a series ofx-ray or fluoroscopic images of a selected surgical site, a system 100to process the surgical images and an overlay of a virtual, augmented,or holographic dimensioned grid 200, with an input device 112 to providemanipulation of the dimensioned grid 200 by a user 202, such as asurgeon. In one embodiment, the electronic display 114 is an electronicdisplay device, such as a computer monitor, or a heads up display, suchas Glass (Google). In another embodiment, the electronic display screen114 is a video fpv goggle. An output to an electronic display 114 isprovided for the user 202 to view the overlay of the series of imagesand the dimensioned grid 200. The augmented reality or holographicdimensioned grid 200 can be manipulated by the user 202 by looking atanatomic landmarks, shown on the electronic display 114 that willfacilitate locking on the correct alignment/placement of surgicaldevice. The system 100 allows the user 202 to see critical workinformation right in their field-of-view using a see through visualdisplay and then interact with it using familiar gestures, voicecommands, and motion tracking. The data can be stored in data storage110. Applications include hip, knee, shoulder, elbow, and anklearthroplasty, trauma fractures and limb deformity correction, spine, andsports medicine procedures such as femoroacetabular impingement/PAO.

While the invention has been described with reference to preferred andexample embodiments, it will be understood by those skilled in the artthat a variety of modifications, additions and deletions are within thescope of the invention, as defined by the following claims.

The invention claimed is:
 1. A computer-implemented method, performed bya processor configured by a readable program code to perform the methodto correct for distortion of an anatomical image captured from an X-rayimaging system comprising the steps of: registering an intra-operativeimage of a patient's anatomy to a digital representation of thedimensioned grid by selecting at least one anatomical landmark toprovide the digital representation of a dimensioned grid with an atleast one grid indicator; calibrating the digital representation of thedimensioned grid to the image of a patient's anatomy to provide acalibrated dimensioned grid; deforming the digital representation of thecalibrated dimensioned grid to correct for the distortion of the imageof a patient's anatomy to generate a deformed calibrated dimensionedgrid image, and intraoperatively displaying the intera-operative imageof the patient's anatomy with reference to the deformed calibrateddimensioned grid image.
 2. The method of claim 1 wherein a deformationfield is computed from the at least one anatomical landmark using athin-plate spline.
 3. The method of claim 2 wherein the step ofregistering the intra-operative image further comprises: aligning the atleast one anatomical landmark of the digital representation of thedimensioned grid with an at least one anatomical landmark in adistortion calibration array image using an affine transformation andthe deformation field.
 4. The method of claim 1 further comprising thestep of: intra-operatively computing a point of interest in the image ofa patient's anatomy with reference to the deformed calibrateddimensioned grid image.
 5. The method of claim 1 further comprising thesteps of; automatically receiving a selection of at least one anatomicallandmark prior to registering an anatomical image to the dimensionedgrid image.
 6. The method of claim 1 further comprising the step ofproviding an at least one visual or auditory instructionintraoperatively to a user.
 7. The method of claim 1 further comprisingthe step of selecting at least one anatomical landmark to provide an atleast one grid indicator for a second anatomical image.
 8. The method ofclaim 7 comprising: adapting the at least one grid indicator for thesecond anatomical image in real time by application of a deformationfield to fit the anatomy of the patient.
 9. The method of claim 1wherein the step of selecting the at least one anatomical landmarkcomprises: receiving an input from a user from a device selected fromthe group consisting of: a mixed reality, an augmented reality and aholographic device.
 10. A digital grid distortion calibration arraybased intra-operative system comprising: a non-transitorycomputer-readable storage medium encoded with computer-readableinstructions which form the application software and a processor toprocess the instructions, wherein the application software includes: adata capture module configured to capture and store a digitalradiographic image of a patient acquired intra-operatively by an imagingdevice; a grid registration module configured to register an anatomicalimage with the digital grid and calibrate a digital representation ofthe digital grid to the image a patient's anatomy to provide acalibrated digital grid; a distortion adaptation module configured todeform the digital representation of the digital grid to correct fordistortion in the anatomical image; and an outcome module configured toprovide at least one intraoperatively instruction to a user andintraoperatively displaying the anatomical image of the patient'sanatomy with reference to the deformed calibrated digital grid image.11. The system of claim 10 further comprising: an auto-segmentationmodule configured to automatically select for a successive anatomicalimage for an at least one anatomical landmark.
 12. The system of claim11 further comprises a calibration grid.
 13. The system of claim 12wherein said calibration grid is selected from the group consisting of:dots and lines.
 14. The system of claim 11 wherein said outcome modulecomprises a digital grid selected from the group consisting of: mixed,augmented reality, and holographic grid.
 15. The system of claim 11wherein said data capture module is configured to obtain data from asensor.
 16. The system of claim 15 wherein said sensor is in electroniccommunication with a grid selected from the group consisting of: a mixedreality, an augmented reality, and a holographic grid.
 17. The system ofclaim 11 wherein said outcome module comprises a display selected fromthe group consisting of: a mixed reality display, an augmented realitydisplay and a holographic display.
 18. The method of claim 1 wherein thestep of deforming the digital representation of the calibrateddimensioned grid is comprised of matching the quantitative warp amountwithin the anatomical medical image with a corresponding deformed imageof a grid.